Three-dimensional (3-D) surface-geometry or shape measurement is useful to build a digital or mathematical representation of an object or environment for which the geometry is unknown. One of the most widely used techniques to obtain the 3-D shape of an unknown object is using structured light. By projecting a known light pattern (stripe, grid, or more complex shape) onto an object, the 3-D coordinates of points on the object surface can be calculated by triangulation from images acquired from another direction. FIG. 1 illustrates this triangulation principle. In the example shown in FIG. 1, a laser line is projected onto a scanned object. The light is scattered from the object surface and the image of the curved line formed on the object is acquired by a camera located at an angle to the laser source. The angle and the position where the scattered light is sensed in the camera sensor array are related. The depth information can be computed from the distorted two-dimensional (2-D) image of the laser light along the detected profile, based on a calibration of the measurement system done earlier. In order to get full range (depth) information, the laser sheet, formed by projecting the laser line, has to be moved across the object or scene. Point-by-point (cf. Parthasarathy, S. et al., “Laser rangefinder for robot control and inspection”, Robot Vision, SPIE, 336, pp. 2-11, 1982; and Rioux, M., “Laser range finder based upon synchronous scanners”, Applied Optics 23(21), pp. 3837-3844, 1984) and line-by-line (cf. Popplestone, R. J. et al., “Forming models of plane-and-cylinder faceted bodies from light stripes”, Proc. Int. Joint Conf. on Artificial Intelligence, pp. 664-668,1975; and Porter II, G. B. and Mundy, J. L., “Noncontact profile sensing system for visual inspection”, Robot Vision, SPIE, 336, pp. 67-76, 1982) scanning use a scanning mechanism equipped with accurate position sensors. Furthermore, they are slow and not practical for real-time 3-D shape measurement.
Multiple-stripe methods speed up the data acquisition, but suffer from a correspondence problem (cf. Boyer, K. and Kak, A., “Color-encoded structured light for rapid active ranging”, IEEE Trans. Pattern Analysis and Machine Intelligence, pp. 14-28, 1987; Chen, C. et al., “Range data acquisition using color structured lighting and stereo vision”. Image and Vision Computing, Vol. 15, pp. 445-456, 1997; and Rusinkiewicz, S. et al., “Real-time 3D Model Acquisition”, Proceedings of Siggraph, pp. 438-446, July 2002) of determining which light stripes in the image correspond with the light stripes actually formed on the object. The correspondence problem can be removed by projecting a multi-frame coded pattern, which carries information of the coordinates of the projected points, without considering geometrical constraints. The coded structured-light approach is an absolute measurement method that encodes all lines in the pattern from left to right, requiring only a small number of images to obtain a full depth-image.
A coded structured-light method called intensity-ratio depth sensing (cf. Carrihill, B. and Hummel, R., “Experiments with the intensity ratio depth sensor”, Computer Vision, Graphics and Image Processing, vol. 32, pp. 337-358. Academic Press, 1985; and Miyasaka, T. and Araki, K., “Development of real time 3-D measurement system using intensity ratio method”, Proc. ISPRS Commission III, Vol. 34, Part 3B, Photogrammetric Computer vision (PCV02), pp. 181-185, Graz, 2002) involves projecting two patterns, a linear grey-level pattern and a constant flat pattern, onto the object and capturing the image of the light pattern formed on the object surface.
An intensity ratio is calculated for every pixel between the two consecutive frames and the 3-D coordinates of each pixel are determined by triangulation. This method has the advantage of fast processing speed, but the accuracy is poor and the problem of ambiguity arises for measuring objects with discontinuous surface shape if the intensity-ratio ramp is repeated (cf. Chazan, G. and Kiryati, N., “Pyramidal intensity-ratio depth sensor”, Technical Report 121, Center for Communication and Information Technologies, Department of Electrical Engineering, Technion, Haifa, Israel, October 1995) to reduce sensitivity to noise.
Full-field optical 3-D shape measurement techniques have been developed to acquire surface-geometry information over a region of a surface rather than just a point or line. Compared with other techniques, it has the benefit of fast measurement speed due to the fact that it does not use scanning to cover the whole object surface.
Moiré interferometry (cf. Takasaki, H., “Moiré topography”, Applied Optics, Vol. 9(6), pp. 1467-1472, 1970 and fringe projection (cf. Creath, K., “Phase-measurement interferometry techniques”, Progress in Optics, Vol. XXVI, E. Wolf, Ed. Elsevier Science Publishers, Amsterdam, pp. 349-393, 1988; Halioua, M. and Liu, H. C. “Optical Three-Dimensional Sensing by Phase Measuring Profilometry”, Optics and Lasers in Engineering, Vol. 11(3), pp. 185-215, 1989; and Greivenkamp, J. E. and Bruning, J. H., Optical Shop Testing, Chapter 4: “Phase Shifting Interferometry”, John Wiley and Sons, Inc., pp. 501-598, 1992) are good representatives of this technique which allows relatively simple image-processing algorithms to extract the 3-D coordinate information, high-speed image grabbing, reliable, quantitative surface measurements, as well as non-contact and noninvasive characteristics, and potential for real-time 3-D shape measurement.
The basis of the moiré method is that a grating pattern is projected onto an object. The projected fringes distort according to the shape of the object. The object surface, together with the projected fringes, is imaged through a grating structure called a reference grating as shown in FIG. 2. The image interferes with the reference grating to form moiré fringe patterns. The moiré fringe patterns contain information about the shape of the object. When the geometry of the measurement system is known, analysis of the patterns then gives accurate descriptions of changes in depth and hence the shape of the object.
Shadow moiré (cf. Takasaki, supra; Meadows, D. M. et al., “Generation of surface contours by moiré pattern” Appl. Opt. 9(4), pp. 942-947, 1970; and Chaing, F. P., “A shadow moiré method with two discrete sensitivities”, Exper. Mech. 15(10), pp. 384-385 1975) is the simplest method of moiré technique for measuring 3-D shapes using a single grating placed in front of the object. The grating in front of the object produces a shadow on the object that is viewed from a different direction through the grating. One advantage of this method is that few or calculations are required to convert image data into profile information of the measured object.
The fringe projection technique discussed earlier, an alternative approach to the moiré method, uses a fringe or grating pattern that is projected onto an object surface and then viewed from another direction. The projected fringes or grating is distorted according to the topography of the object. Instead of using the moiré phenomenon, however, the 3-D surface is measured directly from the fringe projection by triangulation. The image intensity distribution of the deformed fringe pattern or grating is imaged into the plane of a CCD array, then sampled and processed to retrieve the phase distribution through phase extraction techniques, and finally the coordinates of the 3-D object is determined by triangulation.
To increase the measurement resolution, phase measuring interferometry techniques (cf. Takasaki, supra; Creath, supra; and Halioua, supra) have been implemented in moiré and fringe projection methods to extract phase information, among which phase-shifting methods (cf. He, X. Y., et al., “Phase-shifting analysis in moiré interferometry and its application in electronic packaging”, Opt. Eng. 37, pp. 1410-1419, 1998; and Choi, Y. B. and Kim, S. W., “Phase-shifting grating projection moiré topography”, Opt. Eng. 37, pp. 1005-1010, 1998) are the most widely used.
The principle of this technique is that periodic fringe patterns are projected onto an object surface and then viewed from another direction. In general, the minimum number of measurements of the interferogram that will permit reconstruction of the unknown phase distribution is three (cf. Creath, supra), and a sinusoidal fringe pattern is usually used in this technique.
Traditional phase-shifting systems use hardware such as a piezoelectric transducer to produce continuous as well as discrete phase shifts (cf. Creath, supra). In these cases, the accuracy of the extracted phase is limited by the accuracy of the mechanical shifting process. The accuracy also depends on the number of images. More phase steps usually can generate higher accuracy in 3-D shape reconstruction. The trade-off, however, is that longer time is involved both in image acquisition and processing, which is fairly limited for real-time analysis.
Another phase measurement technique is using Fourier transform analysis (cf. Takeda, M., et al., “Fourier Transform Method of Fringe Pattern Analysis for Computer Based Topography and Interferometry”, Journal Opt. Soc. of Am., 72, pp. 156-160, 1982; Kreis, T., “Digital holographic interference-phase measurement using the Fourier transform method”, Journal of the Optical Society of America A. Vol. 3, pp. 847-855, 1986; Freischlad, K. and Koliopoloulos, C., “Fourier description of digital phasemeasuring interferometry,” Journal of the Optical Society of America A. Vol. 7, pp. 542-551, 1990; Malcolm, A. and Burton, D., “The relationship between Fourier fringe analysis and the FFT,” Prypntniewicz R., ed., Laser Interferometry IV: Computer-Aided Interferometry. Proc. of Soc. Photo-Opt. Instr. Eng. 1553. pp. 286-297, 1991; Gorecki, C., “Interferogram analysis using a Fourier transform method for automatic 3D surface measurement”, Pure Appl. Opt., Vol. 1, pp. 103-110, 1992; Gu, J. and Chen, F., “Fast Fourier transform, iteration, and least-squares-fit demodulation image processing for analysis of single-carrier fringe pattern”, Journal of the Optical Society of America A, Vol. 12, pp. 2159-2164, 1995; and Su, X. and Chen, W., “Fourier transform profilometry: a review”, Optics and Lasers in Engineering, 35, pp. 263-284, 2001).
In this method, only one deformed fringe pattern image is used to retrieve the phase distribution. In order to separate the pure phase information in the frequency domain, the Fourier transform usually uses carrier fringes; this poses difficulty in practice in trying to accurately control the frequency of the carrier fringe. Another significant limitation of the Fourier transform technique is its inability to handle discontinuities. Moreover, the complicated mathematical calculation of Fourier transforms is computationally intensive and makes the technique unsuitable for high-speed 3-D shape measurement.
The phase distribution obtained by applying a phase-shifting algorithm is wrapped into the range 0 to 2Π, due to its arctangent feature. A phase unwrapping process (cf. Macy, W. W., “Two-dimensional fringe-pattern analysis”, Appl. Opt. 22, pp. 3898-3901, 1983; Goldstein, R. M. et al., “Satellite Radar Interferometry: Two-Dimensional Phase Unwrapping”, Radio Science, Vol. 23, No. 4, pp. 713-720, 1988; Judge, T. R. and Bryanston-Cross, P. J., “A review of phase unwrapping techniques in fringe analysis”, Optics and Lasers in Engineering, 21, pp. 199-239, 1994; Huntley, J. M. and Coggrave, C. R., “Progress in Phase Unwrapping”, Proc. SPIE Vol. 3407, 1998; and Ghiglia, D. C. and Pritt, M. D., Two-Dimensional Phase Unwrapping: Theory, Algorithms, and Software, Wiley-Interscience, John Wiley and Sons, Inc., 1998) converts the modulo 2Πphase data into its natural range, which is a continuous representation of the phase map. This measured phase map contains the height information of the 3-D object surface (cf. Halioua, supra).
Therefore, a phase-to-height conversion algorithm is usually applied to retrieve the 3-D data of the object. This algorithm is usually related to not only the system setup, but also the relationship between the phase distribution and the height of the object surface. Based on geometric analysis of the measurement system, several phase-to-height mapping techniques (cf. Zhou, W. S. and Su, X. Y., “A direct mapping algorithm for phase-measuring profilometry”, Journal of Modern Optics, Vol. 41, No. 1, pp. 89-94, 1994; Chen, X. et al., “Phase-shift calibration algorithm for phase-shifting interferometry”, Journal of the Optical Society of America A, Vol. 17, No. 11, pp. 2061-2066, November, 2000; Liu, H. et al., “Calibration-based phase-shifting projected fringe profilometry for accurate absolute 3D surface profile measurement”, Optics Communications, Vol. 216, pp. 65-80, 2003; Li, W. et al., “Large-scale three-dimensional object measurement: a practical coordinate mapping and image data-patching method”, Applied Optics, Vol. 40, No. 20, pp. 3326-3333, July, 2001; Guo, H. et al., “Least-squares calibration method for fringe projection profilometry”, Opt. Eng. 44(3), pp. 033603(1-9), March, 2005; and Sitnik, R. et al., “Digital fringe projection system for large-volume 360-deg shape measurement”, Opt. Eng. 41 (2), pp. 443-449, 2002) have been developed, all focused on the accuracy of the measurement. Algorithms that emphasize the speed (cf. Hung, Y. Y. et al., “Practical three-dimensional computer vision techniques for full-field surface measurement”, Opt. Eng. 39 (1), pp. 143-149, 2000; and Zhang, C. et al., “Microscopic phase-shifting profilometry based on digital micromirror device technology”, Applied Optics, Vol. 41, No. 28, pp. 5896-5904, 2002) are also used in some high-speed or real-time systems (cf. Huang, P. S. et al., “High-speed 3-D shape measurement based on digital fringe projection”, Opt. Eng. 42 (1), pp. 163-168, 2003 (“Huang No. 1”); and Zhang, S. and Huang, P. S., “High-resolution, Real-time 3D Shape Acquisition”, IEEE Workshop on real-time 3D sensors and their uses (joint with CVPR 04), Washington D.C., MA, 2004).
More recently, a new digital fringe projection technique for 3-D surface reconstruction has been developed using high-resolution programmable projectors. Compared to traditional fringe projection and laser-interferometric fringe-projection techniques, the computer-generated fringe projection technique has many advantages: (1) any high quality fringe pattern can be precisely and quickly generated by software; (2) the fringe pitch can be easily modified to match the object surface, thus optimizing the range measurement of the object; (3) the phase can be shifted precisely by software according to the specific algorithm without a physical phase shifter; (4) the use of a high and constant brightness and high contrast-ratio projector improves the accuracy of the 3-D shape measurement; and (5) with proper synchronization between the projection and image acquisition, real-time 3-D reconstruction could be achieved.
The most popular methods for computer-generated fringe projection with phase-shifting technique can be roughly divided into grey-scale phase-shifting (cf. Hung, supra; Huang, P. S. and Chiang, F., “Recent advances in fringe projection technique for 3-D shape measurement”, Proc. SPIE, Vol. 3783, pp. 132-142, 1999 (“Huang No. 1”); Fujigaki, M. and Morimoto, Y., “Accuracy of real-time shape measurement by phase-shifting grid method using correlation”, JSME International Journal, Series A, Vol. 43, No. 4, pp. 314-320, 2000; Hu, Q. et al., “Calibration of a three-dimensional shape measurement system”, Opt. Eng. 42(2), pp. 487-493, 2003; Quan, C. et al., “Shape measurement of small objects using LCD fringe projection with phase shifting”, Optics Communications, Vol. 189, pp. 21-29, 2001 and Quan, C. et al., “Shape Measurement by Use of Liquid-Crystal Display Fringe Projection with Two-Step Phase Shifting”, Applied Optics, Vol. 42, No. 13, pp. 2329-2335, 2003 (“Quan No. 2”)), color-encoded phase-shifting (cf. Huang, P. S. et al., “Color-encoded digital fringe projection technique for high-speed three-dimensional surface contouring”, Opt. Eng. 38(6), pp. 1066-1071, 1999; and Pan, J. et al., “Color-encoded digital fringe projection technique for high-speed 3D shape measurement-color coupling and imbalance compensation”, Proc. SPIE, Vol. 5265, pp. 205-212, 2004)46,47 and fringe projection based on the Digital Micromirror Device (DMD) method (cf. Zhang, C., supra; Huang No. 1, supra; Huang, P. S., and Chiang, F. “Method and apparatus for three dimensional surface contouring using a digital video projection system”, U.S. Pat. No. 6,438,272, Aug. 20, 2002; and Huang, P. S. et al., “Method and apparatus for three dimensional surface contouring and ranging using a digital video projection system”, U.S. Pat. No. 6,788,210, Sep. 7, 2004).
The grey-scale phase-shifting method projects a series of phase-shifted sinusoidal grey-scale fringe patterns onto an object and then a camera from another direction captures the images of the perturbed fringe pattern sequentially for processing. Because the phase map acquired directly is limited from −Π to Π, the natural phase distribution of the pixels, which carries the 3-D surface information, is generated by applying phase unwrapping techniques. The 3-D shape information for each pixel is extracted by use of a phase-to-height conversion algorithm. This approach can potentially increase the measurement speed.
Instead of projecting a sinusoidal fringe pattern, Morimoto, Y. et al., “Real-time Shape Measurement by Integrated Phase-Shifting Method” Proc SPIE, Vol. 3744, pp. 118-125, August 199950 and Fujigaki, supra, have proposed an integrated phase-shifting method which projects a rectangular grey-scale pattern multiplied by two weighting functions, respectively. Morimoto built a signal processing board for real-time phase analysis. The system, which records four frames of the deformed grating on the object during one cycle of the phase shift, can obtain the phase distribution every 1/30 seconds. In Fujigaki's method, thirty-two phase-shifted grey-scale images with rectangular distribution are projected and captured to determine the phase difference that corresponds to the height distribution of the object. In the case of worst focus the average phase error is 3% and the maximum phase error is 5% when the object is stable. For a moving object, the error increases linearly with the phase-shifting aberration ratio.
For phase-shifting techniques, sequentially projecting and grabbing images consume time especially for more phase-shift procedures. To further increase the measurement speed, unlike conventional sinusoidal phase-shifting, which uses a minimum of three phase-shifted fringe patterns, a two-step sinusoidal phase-shifting method (cf. Quan No. 2, supra; and Almazan-Cuéllar, S. and Malacara-Hernandez, D., “Two-step phase-shifting algorithm”, Opt. Eng., 42(12), pp. 3524-3531, 2003) and one-frame spatial-carrier sinusoidal phase-shifting method (cf. Kujawinska, M. and Wojciak, J., “Spatial-carrier phase shifting technique of fringe pattern analysis,” Industrial Applications of Holographic and Speckle Measuring Techniques, Proc. SPIE, 1508, pp. 61-67, 1991) for calculation of the phase values has been proposed. Because the phase unwrapping is carried out by use of an arccosine function for two-step sinusoidal phase-shifting, or arctangent function two times for spatial-carrier sinusoidal phase-shifting, the use of these algorithms simplifies the optical system and speeds up the measurement compared to the three-step sinusoidal phase-shifting method. However, the drawback of this method is that the measurement accuracy is lower because the accuracy is dependent on the number of images (of Huzug No. 1, supra; and Morimoto, supra).
The color-encoded method uses a Digital Light Processor (DLP)/Liquid Crystal Display (LCD) projector to project a color-encoded pattern onto the object. Only a single image, which integrates three phase-shifted images (RGB components), is captured by a color CCD camera. The image is then separated into its RGB components, which creates three phase-shifted grey-scale images. These images are used to reconstruct the 3-D object. The problems for this technique include overlapping between the spectra of red, green and blue channels of the color cameras that make the separation of RGB components difficult and intensity imbalance between the separated images of the red, green and blue fringe patterns. The effective separation of the captured image into its RGB components to create three phase-shifted images of the object and compensate the imbalance is non-trivial for this technique.
Fringe projection based on DMD projects a color-encoded fringe pattern onto the object using a DLP projector. Due to the particular features of the DLP projector, the RGB color channels are sequentially projected. With removal of the color filter of the DLP projector and the synchronization between the projection and image acquisition, three grey-scale phase-shifted images are obtained with high speed (cf. Zhang, C., supra, Huang No. 1, supra; and Zhang, S., supra). The 3-D shape of the object is reconstructed using a phase wrapping and unwrapping algorithm and a phase-to-height conversion algorithm. Considering that traditional sinusoidal phase-shifting algorithms involve the calculation of an arctangent function to obtain the phase, which results in slow measurement speed, an improved method, called trapezoidal phase-shifting method (cf. Zhang, S., supra) was proposed for further increasing the processing speed. By projecting three phase-shifted trapezoidal patterns, the intensity ratio at each pixel is calculated instead of the phase. This requires much less processing time.
In fringe-projection techniques, the projected pattern greatly affects the performance. Much processing time is spent on the phase calculation and phase-to-height conversion. To realize real-time 3-D shape measurement, it is not sufficient just to speed up the projection and image acquisition. Designing efficient patterns for fast manipulation are efficient ways of speeding up the entire 3-D measurement process.
The present invention therefore seeks to provide a novel full-field fringe-projection method for 3-D surface-geometry measurement, which is based on digital fringe-projection, intensity ratio, and phase-shifting techniques, and which uses new patterns for fast manipulation to speed up the entire 3-D measurement process.